Dynamic image processing apparatus for aligning frame images obtained by photographing dynamic state of chest based on movement of lung-field region

ABSTRACT

A dynamic image processing apparatus includes: a hardware processor that: extracts a lung-field region from at least one of a plurality of frame images of a chest dynamic image obtained by radiographing a dynamic state of a chest of an examinee; sets a feature point in a position that moves according to a movement of a lung field due to respiration in the lung-field region extracted by the hardware processor; searches a frame image other than a frame image in which the feature point has been set for a corresponding point that corresponds to the feature point set by the hardware processor, and estimates a correspondence relationship of each pixel in the lung-field region among the plurality of frame images in accordance with a positional relationship between the feature point set by the hardware processor and the corresponding point searched for by the hardware processor.

The entire disclosure of Japanese patent Application No. 2017-193141,filed on Oct. 3, 2017, is incorporated herein by reference in itsentirety.

BACKGROUND Technological Field

The present invention relates to a dynamic image processing apparatus.

Description of the Related Art

Conventionally, a technique is known for aligning a subject that isincluded in a plurality of images indicating a change in a state of thesubject. In JP 4493408 B2, for example, a technique is described inwhich, in a chest dynamic image obtained by radiographing a dynamicstate of the chest, a reference image is set between a maximumexhalation image and a maximum inhalation image, and alignment with anadjacent image is sequentially performed on intermediate images betweenthe maximum exhalation image and the maximum inhalation image via thereference image. In JP 4493408 B2, global matching and local matchingare used as processing for obtaining a corresponding position of twoimages in alignment. JP 4493408 B2 describes that, in local matching, anentirety of one image of the two images to be aligned is sectioned intoa large number of template ROIs, a corresponding position in the otherimage of each center pixel of each of the template ROIs is obtained, andcorresponding positions of the other pixels are obtained on the basis ofthe obtained corresponding position.

However, the chest dynamic image includes a structure (such as the ribs)that moves in a direction different from a direction of the movement ofthe lung field due to respiration. Therefore, as an example, when afeature point (a center pixel of a template ROI) for alignment is set inboth the lung field and the ribs, a contradiction is caused in thedirection of movement, and alignment fails to be performed in such a waythat pixels indicating biologically the same position in the lung fieldaccurately correspond to each other. As a result, information relatingto a change in density due to respiration in biologically the sameposition in the lung field fails to be accurately extracted from thechest dynamic image.

SUMMARY

An object of the present invention is to improve the accuracy of theextraction of information relating to a change in density in the lungfield due to respiration in a chest dynamic image.

To achieve the abovementioned object, according to an aspect of thepresent invention, a dynamic image processing apparatus reflecting oneaspect of the present invention comprises a hardware processor that:extracts a lung-field region from at least one of a plurality of frameimages of a chest dynamic image obtained by radiographing a dynamicstate of a chest of an examinee; sets a feature point in a position thatmoves according to a movement of a lung field due to respiration in thelung-field region extracted by the hardware processor; searches a frameimage other than a frame image in which the feature point has been setfor a corresponding point that corresponds to the feature point set bythe hardware processor, and estimates a correspondence relationship ofeach pixel in the lung-field region among the plurality of frame imagesin accordance with a positional relationship between the feature pointset by the hardware processor and the corresponding point searched forby the hardware processor.

BRIEF DESCRIPTION OF THE DRAWINGS

The advantages and features provided by one or more embodiments of theinvention will become more fully understood from the detaileddescription given hereinbelow and the appended drawings which are givenby way of illustration only, and thus are not intended as a definitionof the limits of the present invention:

FIG. 1 illustrates a general configuration of a dynamic image processingsystem according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating photographing control processingperformed by a controller of a console for photographing illustrated inFIG. 1;

FIG. 3 is a flowchart illustrating density change information extractionprocessing performed by a controller of a console for diagnosisillustrated in FIG. 1;

FIG. 4A illustrates an example in which a feature point is set on alung-field contour;

FIG. 4B illustrates an example in which a feature point is set on apulmonary blood vessel;

FIG. 4C illustrates an example in which a feature point is set on thelung-field contour and the pulmonary blood vessel;

FIG. 5 illustrates an example of a correction screen;

FIG. 6A schematically illustrates a positional relationship mapindicating a positional relationship between a feature point and itscorresponding point;

FIG. 6B schematically illustrates a positional relationship mapindicating a positional relationship between all pixels afterinterpolation and their corresponding points;

FIG. 7A illustrates examples of density waveforms before alignment in adynamic image of an examinee having a normal ventilation;

FIG. 7B illustrates density waveforms after alignment according to anembodiment of the present invention in the dynamic image of FIG. 7A;

FIG. 7C illustrates density waveforms after alignment according to anembodiment of the present invention in a dynamic image of an examineehaving an abnormal ventilation;

FIG. 8 illustrates an example in which a motion vector indicating apositional relationship between a feature point and a correspondingpoint is displayed on a reference frame image;

FIG. 9 illustrates an example in which an amount of movement (themagnitude of a vector) in each region, such as the pulmonary apex, thediaphragm, the external thorax, or the internal thorax, in a motionvector still image is indicated using a radar chart; and

FIG. 10 illustrates an example in which only a vertical direction of alung-field region in a certain frame image has been aligned with areference frame image.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, one or more embodiments of the present invention will bedescribed with reference to the drawings. However, the scope of theinvention is not limited to the disclosed embodiments.

[Configuration of Dynamic Image Processing System 100]

First, a configuration according to an embodiment of the presentinvention is described.

FIG. 1 illustrates a general configuration of a dynamic image processingsystem 100 according to an embodiment of the present invention.

As illustrated in FIG. 1, the dynamic image processing system 100 isconfigured in such a way that a photographing device 1 and a console forphotographing 2 are connected to each other via a communication cable orthe like and the console for photographing 2 and a console for diagnosis3 are connected to each other via a communication network NT such as alocal area network (LAN). Respective devices that configure the dynamicimage processing system 100 conform to the digital image andcommunications in medicine (DICOM) standard, and communication among therespective devices is performed according to the DICOM.

[Configuration of Photographing Device 1]

The photographing device 1 is a photographing unit that photographs adynamic state of an organism, such as a change in a form of theexpansion and contraction of the lungs due to respiratory movement orheartbeats. Kymography is a technique for obtaining a plurality ofimages indicating a dynamic state of a subject by repeatedly irradiatingthe subject with radiation such as X-rays in the form of pulses atprescribed time intervals (pulse irradiation) or by continuouslyirradiating the subject with radiation at a low dose rate (continuousirradiation). A series of images obtained in kymography are referred toas a dynamic image. Each of a plurality of images that configure thedynamic image is referred to as a frame image. The embodiment describedbelow will be described using, as an example, a case in which kymographyis performed on the chest by using pulse irradiation.

A radiation source 11 is disposed in a position that faces a radiationdetector 13 across a subject M (an examinee), and irradiates the subjectM with radiation (X-rays) under the control of a radiation irradiationcontroller 12.

The radiation irradiation controller 12 is connected to the console forphotographing 2, and controls the radiation source 11 according toradiation irradiation conditions that are input from the console forphotographing 2 so as to perform radiography. Examples of the radiationirradiation conditions that are input from the console for photographing2 include a pulse rate, a pulse width, a pulse interval, the number ofphotographed frames per photographing, a value of an X-ray tube current,a value of an X-ray tube voltage, the type of an additional filter, andthe like. The pulse rate is the number of times of radiation irradiationper second, and matches the frame rate described later. The pulse widthis a radiation irradiation time period per radiation irradiation. Thepulse interval is a time period from the start of a certain radiationirradiation to the start of the next radiation irradiation, and matchesthe frame interval described later.

The radiation detector 13 is configured by a semiconductor image sensorsuch as a flat panel detector (FPD). The FPD includes, for example, aglass substrate. In a prescribed position on the substrate, a pluralityof detection elements (pixels) are arranged in a matrix form, and eachof the plurality of detection elements (pixels) detects radiation thathas been emitted from the radiation source 11 and has passed through atleast the subject M according to the intensity of the radiation,converts the detected radiation into an electrical signal, and storesthe electrical signal. Each of the pixels is configured to include aswitching unit such as a thin film transistor (TFT). The FPD includes anindirect conversion type FPD that converts X-rays into an electricalsignal by using an optical-to-electrical conversion element via ascintillator and a direct conversion type FPD that directly convertsX-rays into an electrical signal, and any of them may be used.

The radiation detector 13 is provided to face the radiation source 11across the subject M.

A reading controller 14 is connected to the console for photographing 2.The reading controller 14 controls the switching unit of each of thepixels of the radiation detector 13 in accordance with image readingconditions that are input from the console for photographing 2 so as toswitch the reading of the electrical signal stored in each of thepixels, reads the electrical signal stored in the radiation detector 13,and obtains image data. The image data is the frame image. A pixelsignal value of the frame image indicates a density value. The readingcontroller 14 outputs the obtained frame image to the console forphotographing 2. Examples of the image reading conditions include aframe rate, a frame interval, a pixel size, an image size (a matrixsize), and the like. The frame rate is the number of obtained frameimages per second, and matches the pulse rate. The frame interval is atime period from the start of a certain operation to obtain the frameimage to the start of the next operation to obtain the frame image, andmatches the pulse interval.

The radiation irradiation controller 12 and the reading controller 14are connected to each other, and mutually communicate a synchronizingsignal so as to synchronize a radiation irradiation operation and animage reading operation.

[Configuration of Console for Photographing 2]

The console for photographing 2 outputs the radiation irradiationconditions and the image reading conditions to the photographing device1, and controls radiography and an operation to read a radiographicimage that are performed by the photographing device 1. The console forphotographing 2 also displays a dynamic image obtained by thephotographing device 1 in order to cause a photographer such as aradiographer to confirm positioning or to confirm whether the dynamicimage is an image that is suitable for diagnosis.

As illustrated in FIG. 1, the console for photographing 2 is configuredto include a controller 21, a storage 22, an operation unit 23, adisplay 24, and a communication unit 25, and these components areconnected to each other via a bus 26.

The controller 21 is configured by a central processing unit (CPU), arandom access memory (RAM), and the like. The CPU of the controller 21loads and develops a system program or various processing programs thatare stored in the storage 22 into the RAM in accordance with anoperation performed on the operation unit 23, performs various types ofprocessing, such as the photographing control processing describedlater, in accordance with the developed program, and performscentralized control on an operation of each of the components in theconsole for photographing 2, and the radiation irradiation operation andthe reading operation of the photographing device 1.

The storage 22 is configured by a non-volatile semiconductor memory, ahard disk, and the like. The storage 22 stores various programs to beperformed by the controller 21, a parameter that is necessary to performprocessing according to a program, or data such as a processing result.As an example, the storage 22 stores a program for performing thephotographing control processing illustrated in FIG. 2. The storage 22also stores the radiation irradiation conditions and the image readingconditions in association with a region to be examined (in this example,the chest is assumed). The various programs are stored in the form of areadable program code, and the controller 21 sequentially performs anoperation according to the program code.

The operation unit 23 is configured to include a keyboard includingcursor keys, numeric input keys, various function keys, and the like,and a pointing device such as a mouse, and the operation unit 23outputs, to the controller 21, an instruction signal that has been inputby operating keys via the keyboard or operating the mouse. The operationunit 23 may include a touch panel on a display screen of the display 24.In this case, the operation unit 23 outputs, to the controller 21, aninstruction signal that has been input via the touch panel.

The display 24 is configured by a monitor such as a liquid crystaldisplay (LCD) or a cathode ray tube (CRT), and the display 24 displaysan instruction input from the operation unit 23, data, or the like inaccordance with an instruction of a display signal that is input fromthe controller 21.

The communication unit 25 includes a LAN adapter, a modem, a terminaladapter (TA), and the like, and the communication unit 25 controls datatransmission or reception to/from each of the devices that are connectedto the communication network NT.

[Configuration of Console for Diagnosis 3]

The console for diagnosis 3 is a dynamic image processing apparatus thatobtains a dynamic image from the console for photographing 2, performsimage processing on the obtained dynamic image, and displays theprocessed image.

As illustrated in FIG. 1, the console for diagnosis 3 is configured toinclude a controller 31, a storage 32, an operation unit 33, a display34, and a communication unit 35, and these components are connected toeach other via a bus 36.

The controller 31 is configured by a CPU, a RAM, and the like. The CPUof the controller 31 loads and develops a system program or variousprocessing programs that are stored in the storage 32 into the RAM inaccordance with an operation performed on the operation unit 33,performs various types of processing, such as the density changeinformation extraction processing described later, in accordance withthe developed program, and performs centralized control on an operationof each of the components in the console for diagnosis 3. The controller31 functions as an extraction unit, a feature point setting unit, acorresponding point search unit, an estimation unit, a selection unit,and an alignment unit.

The storage 32 is configured by a non-volatile semiconductor memory, ahard disk, and the like. The storage 32 stores various programs such asa program for causing the controller 31 to perform diagnosis assistprocessing, a parameter that is necessary to perform processingaccording to a program, or data such as a processing result. The variousprograms are stored in the form of a readable program code, and thecontroller 31 sequentially performs an operation according to theprogram code.

The storage 32 also stores a dynamic image photographed in the past inassociation with an identification ID, patient information (examineeinformation; for example, a patient ID, the name of a patient (anexaminee), height, weight, age, sex, and the like), examinationinformation (for example, an examination ID, examination date, a regionto be examined (in this example, the chest), a respiratory state, andthe like), and the like.

The operation unit 33 is configured to include a keyboard includingcursor keys, numeric input keys, various function keys, and the like,and a pointing device such as a mouse, and the operation unit 33outputs, to the controller 31, an instruction signal that has been inputby a user operating keys via the keyboard or operating the mouse. Theoperation unit 33 may include a touch panel on a display screen of thedisplay 34. In this case, the operation unit 33 outputs, to thecontroller 31, an instruction signal that has been input via the touchpanel.

The display 34 is configured by a monitor such as an LCD or a CRT, andconducts various displays according to an instruction of a displaysignal that is input from the controller 31.

The communication unit 35 includes a LAN adapter, a modem, a TA, and thelike, and the communication unit 35 controls data transmission orreception to/from each of the devices that are connected to thecommunication network NT.

[Operation of Dynamic Image Processing System 100]

Next, an operation of the dynamic image processing system 100 describedabove according to the present embodiment is described.

(Operations of Photographing Device 1 and Console for Photographing 2)

First, a photographing operation performed by the photographing device 1and the console for photographing 2 is described.

FIG. 2 illustrates photographing control processing performed by thecontroller 21 of the console for photographing 2. The photographingcontrol processing is performed in cooperation between the controller 21and a program stored in the storage 22.

First, a photographer operates the operation unit 23 of the console forphotographing 2 so as to input patient information of an examinee (asubject M) and examination information (step S1).

The radiation irradiation conditions are read from the storage 22 andare set in the radiation irradiation controller 12, and the imagereading conditions are read from the storage 22 and are set in thereading controller 14 (step S2).

An instruction of radiation irradiation that will be issued by operatingthe operation unit 23 is awaited (step S3). At this time, thephotographer disposes the subject M between the radiation source 11 andthe radiation detector 13, and performs positioning. The photographeralso issues an instruction about a respiratory state (for example, quietrespiration) to the examinee (the subject M). At a point in time atwhich photographing preparation is finished, the photographer operatesthe operation unit 23, and inputs a radiation irradiation instruction.

When the radiation irradiation instruction is input via the operationunit 23 (step S3; YES), a photographing start instruction is output tothe radiation irradiation controller 12 and the reading controller 14,and kymography is started (step S4). Stated another way, radiation isemitted from the radiation source 11 at pulse intervals that have beenset in the radiation irradiation controller 12, and a frame image isobtained by the radiation detector 13.

When a prescribed number of frames are photographed, the controller 21outputs an instruction to finish photographing to the radiationirradiation controller 12 and the reading controller 14, and thephotographing operation is terminated. The number of frames to bephotographed is the number of frames in which at least a singlerespiratory cycle can be photographed.

The frame image obtained in photographing is sequentially input to theconsole for photographing 2, is stored in the storage 22 in associationwith a number (a frame number) indicating a photographing order (stepS5), and is displayed on the display 24 (step S6). The photographerconfirms positioning and the like by using a displayed dynamic image,and determines whether an image suitable for diagnosis has been obtainedin photographing (photographing OK) or rephotographing will be performed(photographing NG). The photographer operates the operation unit 23, andinputs a determination result.

When a determination result indicating photographing OK is input byperforming a prescribed operation on the operation unit 23 (step S7;YES), information, such as the identification ID for identifying adynamic image, the patient information, the examination information, theradiation irradiation conditions, the image reading conditions, or thenumber (the frame number) indicating the photographing order, is addedto each of a series of frame images that have been obtained inkymography (for example, the information is written to a header area ofimage data in the DICOM form), and each of the series of the frameimages is transmitted to the console for diagnosis 3 via thecommunication unit 25 (step S8). Then, this processing is terminated.When a determination result indicating photographing NG is input byperforming a prescribed operation on the operation unit 23 (step S7;NO), a series of frame images stored in the storage 22 are deleted (stepS9), and this processing is terminated. In this case, rephotographingwill be performed.

[Operation of Console for Diagnosis 3]

Next, an operation of the console for diagnosis 3 is described.

In the console for diagnosis 3, when a series of frame images of adynamic image are received from the console for photographing 2 via thecommunication unit 35, the received series of frame images of thedynamic image are stored in the storage 32 in association with theidentification ID, the patient information, the examination information,and the like. In addition, when a single dynamic image is selected fromdynamic images stored in the storage 32 by using the operation unit 33and an instruction to extract density change information is issued, thedensity change information extraction processing illustrated in FIG. 3is performed in cooperation between the controller 31 and a programstored in the storage 32. The density change information extractionprocessing is described below with reference to FIG. 3.

First, a single frame image is selected as a reference frame image froma plurality of frame images that configure a dynamic image (step S11).The reference frame image is a frame image that is used as a referencefor alignment among the plurality of frame images, and any of theplurality of frame images may be used as the reference frame image. Asan example, a frame image in a preset respiration phase (for example, amaximum exhalation phase or a maximum inhalation phase) may beautomatically selected, or may be selected by a user operating theoperation unit 33.

Then, a lung-field region is extracted from the reference frame image(step S12).

The lung-field region may be extracted by using any publicly knownmethod. As an example, a threshold is obtained from a histogram of asignal value (a density value) of each pixel in the frame image byperforming determination analysis, and a region of a signal having avalue greater than the threshold is primarily extracted as a lung-fieldregion candidate. By performing edge detection near a boundary of theprimarily extracted lung-field region candidate and extracting, alongthe boundary, a point in which an edge becomes the maximum in each smallblock near the boundary, a boundary of the lung-field region can beextracted.

A feature point is set in the extracted lung-field region (step S13).

In step S13, the feature point is set in a position that moves accordingto the movement of the lung field due to respiration (a position thatreflects the movement of the lung field) in the lung-field regionextracted from the reference frame image. In this processing, acorresponding point (biologically the same point) of each point in thelung field is obtained in each of the frame images, and the densitychange information is extracted. When the feature point is set on astructure that moves in a direction that is different from a directionof the movement of the lung field, processing accuracy is reduced insearching for or estimating the corresponding point. Therefore, in stepS13, the feature point is set in the point that moves according to themovement of the lung field due to respiration (the position thatreflects the movement of the lung field), and the feature point is notset in a position that moves differently from the movement of the lungfield due to respiration, such as a position in which the ribs exist.

As an example, the feature point is set on a lung-field contour, asillustrated in FIG. 4A. This is because the lung-field contour isassumed to most accurately reflect the movement of the lung field due torespiration. A lung-field contour near the heart strongly reflects themovement of heartbeats rather than a movement due to respiration.Therefore, it is preferable that the feature point be set in a positionexcluding a portion that overlaps a heart region of the lung-fieldcontour. The heart region can be extracted by using, for example,template matching using a template image of the heart.

As another example, the feature point may be set on a pulmonary bloodvessel, as illustrated in FIG. 4B. This is because pulmonary bloodvessels exist in the lung-field region and move according to themovement of the lung field due to respiration and therefore thepulmonary blood vessels are assumed to reflect the movement of the lungfield. In processing for extracting the pulmonary blood vessels, amethod for extracting a linear structure by using a Hessian matrix canbe used, for example (see, for example, Qiang Li, “Selective enhancementfilters for nodules, vessels, and airway sails in two- andthree-dimensional CT scans”, MEDICAL PHYSICS, SEPTEMBER 2003). It isdifficult to recognize/trace the arterioles having a diameter that isless than or equal to 0.3 mm, and the arterioles do not reflect themovement of the lung field in comparison with thick blood vessels.Accordingly, it is preferable that the feature point be set on bloodvessels excluding the arterioles from the extracted pulmonary bloodvessels, namely, on the aortas, the arteries, the venae cavae, and/orthe venae.

As yet another example, the feature point may be set on the lung-fieldcontour (a lung-field contour excluding a portion that overlaps theheart region is preferable) and the pulmonary blood vessel (the aortas,the arteries, the venae cavae, and/or the venae are preferable), asillustrated in FIG. 4C.

Information (such as only the lung-field contour, only the pulmonaryblood vessel, or the lung-field contour+the pulmonary blood vessel)relating to the setting of the feature point in a previous dynamic imagemay be stored in the storage 32 in association with the patientinformation, and the feature point may be set in the same portion as aportion in which the feature point has been set in the past on the basisof information relating to the setting in the past of the feature pointof an examinee.

Feature-point position information (coordinates) set in a previousdynamic image may be stored in the storage 32 in association with thepatient information, and the feature point may be set in the sameposition (coordinates) as a position in which the feature point is setin the previous dynamic image of an examinee.

A user may be allowed to specify a structure on which the feature pointwill be set by using the operation unit 33. The user may be allowed tospecify, for example, only the lung-field contour, only the pulmonaryblood vessel, the lung-field contour+the pulmonary blood vessel, or thelike. The user may be allowed to perform weighting (weighting to be usedin the process of step S15) on a structure on which the feature pointwill be set.

In order to improve a degree of freedom and accuracy in setting thefeature point, it is preferable that the controller 31 display, on thedisplay 34, a correction screen 341 on which the user corrects theposition of an automatically set feature point, and correct the positionof the feature point according to the user's operation via the operationunit 33 (a correction unit).

FIG. 5 illustrates an example of the correction screen 341. Asillustrated in FIG. 5, on the correction screen 341, a reference frameimage 341 a is displayed in which a mark is added in the position of thefeature point. The position of the feature point can be corrected, forexample, by a user specifying a feature point for which a position isdesired to be corrected by using the mouse or the like of the operationunit 33 and dragging (moving) the feature point to a position aftercorrection.

It is preferable that the correction screen 341 have a function ofmagnifying and displaying the lung-field region. By doing this, theaccuracy of correction and the workability of correction can beimproved.

A plurality of feature points may be enabled to be grouped and moved(corrected). By doing this, the efficiency of a correction task can beimproved.

Before this processing, the processes of step S14 to step S16 describedbelow may be performed on a small number of feature points or a reducedimage so as to generate a preview image of an alignment image anddisplay the preview image on the display 34. By doing this, timerequired to re-perform processing can be reduced.

Next, each of the frame images excluding the reference frame image issearched for a corresponding point that corresponds to the feature point(step S14).

As a method for searching for the corresponding point on each of theframe images excluding the reference frame image, the correspondingpoint may be directly searched for between the reference frame image andeach of the frame images, or the corresponding point may be sequentiallysearched for between each adjacent frame images starting from thereference frame image.

A case is described in which the corresponding point is sequentiallysearched for between each adjacent frame images starting from thereference frame image. Assume, for example, that the total number offrame images is N and the reference frame image is an i-th frame image.First, an (i+1)th frame is searched for a corresponding point thatcorresponds to a feature point in the i-th frame. Then, a correspondingpoint in an (i+2)th frame is searched for by using the correspondingpoint that has been searched for in the (i+1)th frame as a reference,and corresponding points are similarly searched for in the next frame toan N-th frame. Similarly, a corresponding point that corresponds to thefeature point in the i-th frame is searched for in an (i−1)th frame.Then, a corresponding point in an (i−2)th frame is searched for by usingthe corresponding point that has been searched for in the (i−1)th frameas a reference, and corresponding points in the next frame to a 1stframe are similarly searched for. In this processing, a motion vectorfrom the feature point in the reference frame image to a correspondingpoint in each of the frame images can be calculated by adding all of themotion vectors from the reference frame image to each of the frameimages. As an example, when the reference frame image is the i-th frameand a motion vector from the feature point in the reference frame imageto a corresponding point in an (i+3)th frame is desired to becalculated, a motion vector from the feature point in the i-th frame toa corresponding point in the (i+1)th frame, a motion vector from thecorresponding point in the (i+1)th frame to a corresponding point in the(i+2)th frame, and a motion vector from the corresponding point in the(i+2)th frame to a corresponding point in the (i+3)th frame may beadded.

As a method for searching for a corresponding point in each of the frameimages, as an example, template matching can be used in which a ROIhaving a prescribed size with each of the feature points in thereference frame image (in the case of search between adjacent frameimages, each of corresponding points of feature points that have beensearched for in a frame image as a reference) as a center is used as atemplate image. A degree of similarity in template matching can becalculated using, for example, sum of squared difference (SSD), sum ofabsolute difference (SAD), a cross-correlation coefficient, or the like.

A corresponding point that corresponds to each of the feature points inthe reference frame image may be searched for in each of the frameimages by using, for example, optical flow. Examples of optical flowinclude the Lucas-Kanade method of a sparse type, the Horn-Schunckmethod and the Gunnar-Farneback method of a dense type, and the like.Any of the methods may be used.

The position of a corresponding point that corresponds to each of thefeature points in the reference frame image may be searched for in eachof the frame images by using a previous corresponding point searchmodel. As an example, a corresponding point search result in a previousdynamic image may be stored as a corresponding point search model in thestorage 32 in association with the patient information and/or theexamination information, and the position of a corresponding point thatcorresponds to a feature point may be obtained using a previouscorresponding point search model of an examinee, or a previouscorresponding point search model of another examinee whose informationrelating to a prescribed item (such as age, sex, height, weight,nationality, or a respiratory state in photographing) of the examineeinformation and/or the examination information is similar to (matches orhas a difference within a prescribed range from) information relating tothe examinee. As an example, a corresponding point that is locatedclosest to the feature-point position set in step S13 in a correspondingpoint search model may be employed as a corresponding point, or acorresponding point that corresponds to a feature point in thecorresponding point search model may be employed as a correspondingpoint with no change.

Then, for each of the frame images, a positional relationship map isgenerated in which a motion vector indicating a positional relationshipbetween a feature point set in the reference frame image and itscorresponding point is mapped, and interpolation is performed on thegenerated positional relationship map such that a corresponding point ofeach of the pixels (points other than the feature point) in each of theframe images is estimated (step S15).

FIG. 6A schematically illustrates a positional relationship mapindicating a positional relationship between a feature point and itscorresponding point. FIG. 6B schematically illustrates a positionalrelationship map indicating a positional relationship between each ofthe pixels after interpolation and its corresponding point. Examples ofinterpolation include spline interpolation and polynomial approximation.

In a case in which weighting is performed on a feature point, acorresponding point of each of the pixels is estimated on the basis of apositional relationship between a heavily weighted feature point and itscorresponding point in step S15. By doing this, the accuracy of theestimation of a corresponding point can be improved.

In each of the frame images, an alignment image (referred to as analignment dynamic image) is generated (step S16). In the alignmentdynamic image, lung-field regions in frame images other than thereference frame image have been aligned with a lung-field region in thereference frame image on the basis of the generated positionalrelationship map of all of the pixels.

In the present embodiment, only a point that moves according to themovement of the lung field due to respiration is set as the featurepoint. Therefore, as the alignment dynamic image generated according tothe present embodiment, an image can be obtained in which a structurethat moves differently from the lung field, such as the ribs, isdistorted (becomes unclear) due to misalignment, but the positions ofthe lung-field contour and the pulmonary blood vessels are fixed amongthe frame images, and are clear.

Next, the density change information is extracted on the basis of eachof the aligned frame images (step S17).

As an example, each of the frame images is divided into a plurality ofsmall regions (for example, small regions of 0.4 to 4 square cm), arepresentative value (such as a mean value, a median, a maximum value,or a minimum value) of density values of a plurality of pixels includedin each of the small regions is calculated for each of the smallregions, and a density waveform indicating a temporal change in thecalculated density value is extracted as the density change information.

In the present embodiment, the alignment dynamic image is generated onthe basis of a corresponding point between frame images, and the densitywaveform is extracted from the generated alignment dynamic image.However, the density waveform can be extracted by tracing a change indensity of a corresponding point in each of the frame images withoutgenerating the alignment image.

Next, the extracted density waveform is displayed on the display 34(step S18), and the density change information extraction processing isterminated.

FIG. 7A schematically illustrates density waveforms that are extractedin positions A and B (position A is a position onto which the ribs arenot superimposed; position B is a position onto which the ribs aresuperimposed) of a dynamic image before alignment of an examinee havinga normal ventilation. FIG. 7B schematically illustrates densitywaveforms that are extracted in positions A and B of a dynamic imageafter alignment according to the present embodiment of the same examineeas the examinee of FIG. 7A. FIG. 7C schematically illustrates densitywaveforms that are extracted in positions A and B of a dynamic imageafter alignment according to the present embodiment of an examineehaving an abnormal ventilation.

In the case of the examinee having a normal ventilation, the phase of adensity waveform indicating a change in density in a temporal directionof the lung field due to respiration is expected to substantially matchthe phase of a waveform (referred to as a diaphragm position waveform)that is obtained by inverting a waveform indicating a temporal change inthe position of the diaphragm (for example, coordinates in a case inwhich the position of the diaphragm in a quiet inhalation phase (at atime when the position of the diaphragm is located in the lowestposition) is set as the origin). However, when the feature point is alsoset in a structure that moves differently from the movement of the lungfield due to respiration, alignment is performed, and a density waveformis extracted from the obtained image, as in a conventional technique, ina region, such as position B of FIG. 7A, that is located near a regionthat overlaps the structure that moves differently from the movement ofthe lung field due to respiration, such as the ribs, a phase shift fromthe diaphragm position waveform (in an extreme case, a reverse phase) isgenerated due to an influence of these movements. In contrast, when adensity waveform is extracted from an alignment image of an examineehaving a normal ventilation that has been generated using the techniqueaccording to the present embodiment, a density waveform in which a phaseshift from the diaphragm position waveform is reduced can be obtained,as illustrated in FIG. 7B. In addition, in a case in which a densitywavelength is extracted from the alignment image generated in thepresent embodiment, it can be determined that an abnormality is highlylikely to have occurred in a region in which a density waveform forwhich a phase deviates from a phase of the diaphragm position waveformis extracted, as illustrated in FIG. 7C.

As information that assists the diagnosis of a doctor, the controller 31may also display the information described below on the display 34.

(1) As an example, the motion vector described above that indicates thepositional relationship between the feature point and the correspondingpoint is added to a frame image of the dynamic image, and is displayedon the display 34. By doing this, a doctor can easily grasp whether themovement of the lung field is normal or abnormal. (1-1) As an example,as illustrated in FIG. 8, the feature point set in step S13 and a motionvector indicating a positional relationship between the feature pointand its corresponding point are displayed on the reference frame image.As the motion vector, a motion vector indicating a positionalrelationship between the feature point and a corresponding point that islocated farther from the feature point is displayed, for example. (1-2)As an example, a motion vector between the feature point in thereference frame image and its corresponding point is added to each ofthe frame images, and a dynamic image is displayed on the display 34.

The image displayed in (1-1) described above is referred to as a motionvector still image, and the image displayed in (1-2) is referred to as amotion vector dynamic image.

(2) Motion vector still images or motion vector dynamic images of anexaminee and a normal person are displayed side by side on the display34. By doing this, a doctor can easily grasp whether the movement of thelung field is normal or abnormal. (3) A motion vector still image or amotion vector dynamic image that only indicates a motion vectorindicating an orientation specified by a user is displayed on thedisplay 34. (4) A motion vector still image or a motion vector dynamicimage that only indicates a motion vector in a region in which there isa large difference between a left-hand lung field and a right-hand lungfield is displayed on the display 34. (5) Motion vector information in aposition specified by a user is displayed in time series on the display34 in the form of a graph (such as a line graph). (6) Previous motionvector still images or motion vector dynamic images have been stored inthe storage 32 in association with the patient information, and motionvector still images or motion vector dynamic images of an examineeduring a prescribed period are read from the storage 32, and aredisplayed side by side on the display 34. This enables follow-upobservation. (7) Previous motion vector still images or motion vectordynamic images have been stored in the storage 32 in association with adisease name (such as COPD or interstitial pneumonia), the storage 32 issearched for images that include a motion vector that is similar to amotion vector in a motion vector still image or a motion vector dynamicimage that has been currently generated, and the images that have beensearched for are displayed on the display 34 together with the diseasename in order of a greater degree of similarity of the motion vector. Bydoing this, the name of a disease that has a movement similar to themovement of the lung field of an examinee can be easily grasped. (8) Thealignment image is displayed on the display 34. (9) One of the left-handlung field and the right-hand lung field is laterally inverted in thealignment image, the motion vector still image, or the motion vectordynamic image, and the orientations of the left-hand and right-hand lungfields are aligned and displayed. (10) A waveform indicating a temporalchange in a motion vector (for example, the magnitude of the motionvector) of the diaphragm is generated, and is displayed.

When the motion vector is displayed in (1) to (9) described above, thecolor of the motion vector may be changed according to the magnitude ofthe motion vector. As an example, a look up table (LUT) in which themagnitude and the color of the motion vector are associated with eachother may be stored in the storage 32, and the motion vector may bedisplayed in a color according to the magnitude of the motion vector inaccordance with the LUT.

The thickness of the motion vector may be changed according to themagnitude of the motion vector.

By doing this, a doctor can more easily grasp a portion that has a largemovement amount and a portion that has a small movement amount.

The color of the motion vector may be changed according to the directionof the motion vector. As an example, a look up table (LUT) in which thedirection and the color of the motion vector are associated with eachother may be stored in the storage 32, and the motion vector may bedisplayed in a color according to the direction of the motion vector inaccordance with the LUT. By doing this, a doctor can more easily graspthe direction of a movement.

As information that assists a doctor to provide an explanation to apatient, the controller 31 may display information (11) to information(17) described below on the display 34.

(11) Motion vector still images or motion vector dynamic images of anormal person and a disease patient are displayed simultaneously (sideby side or superimposedly). (12) A previous motion vector still image ormotion vector dynamic image of an examinee has been stored in thestorage 32, and the previous motion vector still image or motion vectordynamic image and a motion vector still image or motion vector dynamicimage that has been generated in a current examination of the examineeare displayed side by side or superimposedly in order to easily conductfollow-up observation. (13) A change portion (the magnitude andorientation of the motion vector) between a previous motion vector stillimage or motion vector dynamic image and a motion vector still image ormotion vector dynamic image that has been generated in a currentexamination of an examinee is emphasized (for example, the changeportion is changed in color), and these images are displayed side byside or superimposedly in order to easily conduct follow-up observation.(14) When a motion vector still image or motion vector dynamic image ofan examinee is displayed, a numerical value (a numerical value of achange amount) and a descriptive text (for example, “move in ∘∘direction by ΔΔ mm”) of a motion vector are additionally displayed. Thisenables a movement amount of the lung field to be quantitativelydisplayed. (15) A disease that has a motion vector that is similar tothe motion vector of an examinee is searched for, and the name of thedisease and a difference amount between the motion vector of theexaminee and the motion vector of the disease are displayed on thedisplay 34. By doing this, the examinee and a disease that has amovement of the lung field similar to the movement of the lung field ofthe examinee can be easily compared with each other. (16) A currentmotion vector still image or dynamic image of an examinee and a motionvector still image or dynamic image in a case in which a disease will beameliorated (a motion vector still image or dynamic image of prognosticprediction) are displayed simultaneously (side by side orsuperimposedly). (17) As illustrated in FIG. 9, an amount of movement(the magnitude of a vector) in each region, such as the pulmonary apex,the diaphragm, the external thorax, or the internal thorax, in a motionvector still image of an examinee is displayed using a radar chart.

As described above, the controller 31 of the console for diagnosis 3extracts a lung-field region from at least one of a plurality of frameimages of a chest dynamic image, sets a feature point in a position thatmoves according to the movement of the lung field due to respiration inthe extracted lung-field region, and searches frame images other thanthe frame image in which the feature point has been set for acorresponding point that corresponds to the set feature point. Acorrespondence relationship of each pixel in the lung-field region isestimated among the plurality of frame images of the chest dynamic imageon the basis of a positional relationship between the set feature pointand the corresponding point that has been searched for.

Accordingly, the feature point is set in a position that moves accordingto the movement of the lung field due to respiration in the lung-fieldregion. Therefore, a contradiction is not caused in the direction ofmovement in contrast to a case in which the feature point is set in astructure (such as the ribs) that moves in a direction that is differentfrom the direction of the movement of the lung field due to respiration,and a corresponding point is searched for or estimated. This enablespixels indicating biologically the same position in the lung field to beaccurately made to correspond to each other. As a result, the accuracyof the extraction of information relating to a change in density in thelung field due to respiration can be improved.

The descriptive contents of the embodiment above are a preferableexample of the present invention, and the present invention is notlimited to this.

As an example, the embodiment above has been described using, as anexample, a case in which one reference frame image is selected. However,a plurality of reference frame images may be selected. As an example, ina dynamic image in which a plurality of respiratory cycles arephotographed, a frame image in each quiet exhalation phase may beselected as the reference frame image. A configuration may be employedin which a user interface (such as an operation screen) is included thata user uses to specify each of the reference frame images and a range offrame images to be aligned with each of the reference frame images byoperating the operation unit 33.

In the embodiment above, it has been described that a lung-field regionis extracted from a reference frame image, a point that moves accordingto the movement of the lung field due to respiration in the extractedlung-field region is set as a feature point, and another frame image issearched for a point that corresponds to the set feature point. However,the lung-field region may be extracted from each of the frame images,the feature point may be set in the lung-field region in each of theframe images, and another frame image may be searched for a point thatcorresponds to the set feature point in each of the frame images.

The controller 31 may serve as a suppression unit so as to perform bonesuppression processing (see, for example, WO2015/157067) for suppressinga signal component resulting from a bone on each of a plurality of frameimages in a chest dynamic image, and may perform the processes of stepS13 and the subsequent steps in the density change informationextraction processing described above by using frame images in which thesignal component resulting from the bone has been suppressed. By doingthis, the accuracy of the recognition/tracing of pulmonary blood vesselsis improved. In addition, a change in density due to bones in thelung-field region can be suppressed from being generated, and theaccuracy of density analysis is improved.

The lung-field region may be extracted before or after the bonesuppression processing. When at least the setting of a feature point anda search for a corresponding point that corresponds to the feature pointare performed using frame images after the bone suppression processing,alignment can be accurately performed.

A movement in a vertical direction of the lung field (principally themovement of the diaphragm) is very large with respect to movements inother directions, and when a search for the corresponding pointaccording to the embodiment described above is conducted in themovements in the vertical direction and the other directions, theaccuracy of the search may be reduced. Accordingly, the controller 31may extract in advance a lung-field region from each of the frameimages, may align, in the vertical direction, the lung-field regions inthe respective frame images with each other (a vertical-directionalignment unit), and may perform the processes of step S13 and thesubsequent steps in FIG. 3 on a dynamic image on which alignment in thevertical direction has been performed.

As an example, a single reference frame image is selected from aplurality of frame images of a dynamic image, and lung-field regions inthe other frame images are aligned with a lung-field region in thereference frame image only in the vertical direction. Stated anotherway, the shapes of the lung fields in the other frame images aremagnified/reduced in the vertical direction so as to match the shape ofthe lung field in the reference frame image (see FIG. 10). The processesof step S13 and the subsequent processes in FIG. 3 are performed byusing each of the frame images in which the lung-field region has beenaligned in the vertical direction.

As a method for aligning the lung-field regions in the other frameimages with the lung-field region in the reference frame image only inthe vertical direction, a method can be used, for example, forestimating a transformation matrix by using a three-point method amongframe images to be aligned and aligning lung-field regions in thevertical direction by using the estimated matrix according to (a) to (c)described below. (a) Reference points (for example, three point, amidpoint of left-hand and right-hand pulmonary apexes, the apex of aright-hand diaphragm, and the apex of a left-hand diaphragm) areextracted from each of the frame images. (b) A transformation matrix isestimated in such a way that the positions of three reference pointsextracted from each of the frame images match the positions of threereference points extracted from the reference frame image. (c) Alung-field region in each of the frame images is aligned in the verticaldirection with a lung-field region in the reference frame image by usingthe estimated transformation matrix.

As the reference point, four points, the left-hand and right-handpulmonary apexes, the apex of the right-hand diaphragm, and the apex ofthe left-hand diaphragm, may be extracted, and the transformation matrixmay be estimated by using the four extracted points as the referencepoint.

As described above, a lung-field region is extracted in advance fromeach of the frame images, the lung-field regions in the respective frameimages are aligned with each other in the vertical direction, and thesetting of a feature point, a search for a corresponding point, andalignment are performed on a dynamic image on which alignment in thevertical direction has been performed. This enables the extraction(alignment) of a corresponding point with accuracy in all directions.

As an example, in the description above, an example has been disclosedin which a hard disk, a non-volatile semiconductor memory, or the likeis used as a computer readable medium storing a program according to anembodiment of the present invention. However, the present invention isnot limited to this example. A portable recording medium such as aCD-ROM can be employed as another computer readable medium. In addition,a carrier wave can be employed as a medium that provides data of aprogram according to an embodiment of the present invention via acommunication line

Changes can be appropriately made to detailed configurations anddetailed operations of respective devices that configure the dynamicimage processing system without departing from the spirit of the presentinvention.

Although embodiments of the present invention have been described andillustrated in detail, the disclosed embodiments are made for purposesof illustration and example only and not limitation. The scope of thepresent invention should be interpreted by terms of the appended claims.

What is claimed is:
 1. A dynamic image processing apparatus comprising: a hardware processor that: extracts a lung-field region from at least one of a plurality of frame images of a chest dynamic image obtained by radiographing a dynamic state of a chest of an examinee; sets a feature point in a position that moves according to a movement of a lung field due to respiration in the lung-field region extracted by the hardware processor; searches a frame image other than a frame image in which the feature point has been set for a corresponding point that corresponds to the feature point set by the hardware processor; and estimates a correspondence relationship of each pixel in the lung-field region among the plurality of frame images in accordance with a positional relationship between the feature point set by the hardware processor and the corresponding point searched for by the hardware processor.
 2. The dynamic image processing apparatus according to claim 1, wherein the hardware processor sets the feature point in a position of a lung-field contour in the lung-field region extracted by the hardware processor.
 3. The dynamic image processing apparatus according to claim 2, wherein the hardware processor sets the feature point in a position that does not overlap a heart region of the lung-field contour.
 4. The dynamic image processing apparatus according to claim 1, wherein the hardware processor sets the feature point in positions of pulmonary blood vessels in the lung-field region extracted by the hardware processor.
 5. The dynamic image processing apparatus according to claim 4, wherein the hardware processor sets the feature point in aortas, arteries, venae cavae, or venae of the pulmonary blood vessels.
 6. The dynamic image processing apparatus according to claim 1, wherein the hardware processor searches for the corresponding point that corresponds to the feature point by using template matching or optical flow.
 7. The dynamic image processing apparatus according to claim 1, further comprising: a storage that stores a corresponding point search model in association with examinee information and/or examination information, the corresponding point search model indicating a corresponding point search result in a previous chest dynamic image, wherein the hardware processor searches for the corresponding point that corresponds to the feature point in accordance with the corresponding point search model stored in the storage of the examinee or an examinee that is similar to the examinee in a prescribed item of the examinee information and/or the examination information.
 8. The dynamic image processing apparatus according to claim 1, wherein the hardware processor estimates the correspondence relationship of each of the pixels in the lung-field region among the plurality of frame images by generating, for each of the plurality of frame images, a positional relationship map in which a motion vector is mapped, the motion vector indicating a positional relationship between the feature point and the corresponding point of the feature point, and by estimating a position of a corresponding point that corresponds to each pixel other than the feature point in the frame image in which the feature point has been set by interpolating the positional relationship map that has been generated.
 9. The dynamic image processing apparatus according to claim 1, wherein the hardware processor performs bone suppression processing on the plurality of frame images of the chest dynamic image, and the hardware processor at least sets the feature point and searches for the corresponding point by using the plurality of frame images on which the bone suppression processing has been performed.
 10. The dynamic image processing apparatus according to claim 1, wherein the hardware processor selects a single reference frame image from the plurality of frame images, the hardware processor extracts the lung-field region from the single reference frame image, and the hardware processor sets the feature point in the position that moves according to the movement of the lung field due to the respiration in the lung-field region of the single reference frame image.
 11. The dynamic image processing apparatus according to claim 1, wherein the hardware processor: selects a single reference frame image from the plurality of frame images; and extracts the lung-field region from each of the plurality of frame images, and aligns a vertical direction of the lung-field region of each of the plurality of frame images with the vertical direction of the lung-field region of the single reference frame image, and the hardware processor at least sets the feature point and searches for the corresponding point by using the plurality of frame images in which the vertical direction of the lung-field region has been aligned.
 12. The dynamic image processing apparatus according to claim 11, wherein the hardware processor obtains positions of reference points in a diaphragm and a pulmonary apex in accordance with the lung-field region extracted from each of the plurality of frame images, estimates a transformation matrix for aligning the vertical direction of the lung-field region of each of the plurality of frame images with the vertical direction of the lung-field region of the single reference frame image in accordance with the obtained positions of the reference points, and aligns the vertical direction of the lung-field region of each of the plurality of frame images with the vertical direction of the lung-field region of the single reference frame image by using the estimated transformation matrix.
 13. The dynamic image processing apparatus according to claim 10, wherein the hardware processor generates an alignment image in accordance with the correspondence relationship of each of the pixels in the lung-field region among the plurality of frame images, the correspondence relationship being estimated by the hardware processor, the alignment image being obtained by aligning the lung-field region of each of the plurality of frame images with the lung-field region of the single reference frame image.
 14. The dynamic image processing apparatus according to claim 1, wherein the hardware processor; selects a single reference frame image from the plurality of frame images; and generates an alignment image in accordance with a corresponding position of each of the pixels in the lung-field region among the plurality of frame images, the corresponding position being estimated by the hardware processor, the alignment image being obtained by aligning the lung-field region of each of the plurality of frame images with the lung-field region of the single reference frame image.
 15. The dynamic image processing apparatus according to claim 1, wherein the hardware processor causes a user to correct the feature point set by the hardware processor.
 16. The dynamic image processing apparatus according to claim 1, further comprising: a display that displays the chest dynamic image, the chest dynamic image being added with the positional relationship between the feature point that has been set and the corresponding point that has been searched. 